A paper medium has been used to read a document. Nowadays, opportunities to read documents of electronic media have increased with the increase in performance of computers and development of the Internet. An advantage of paper media is that the document may be read in a large size, paper media are low in cost, paper media may be overviewed to grasp the entire text, and so forth. An advantage of electronic media is that dynamically-changing information, such as information on the Internet and moving images, may be displayed, electronic media are easy to carry, and so forth. Then, a new interface that improves a utility value of a document by causing paper media to cooperate with electronic media while utilizing the advantage of paper media has been developed.
Such an interface has a function to pick up an image of a document of a paper medium existing before a user using a camera fixed at an arbitrary position or a movable camera to acquire a document image, and then to display additional information related to the document. Recently, as a technique to display the additional information, an information presenting method using augmented reality (AR) in which additional information is displayed on a document image or a projected paper document in a superimposed manner has been proposed. With the information presenting method, it becomes possible to associate the additional information with a display position on a text image, whereby a paper medium and an electronic medium may be cooperated.
As a related art, for example, a method for picking up an image of a document of a paper medium using a portable terminal, retrieving an electronic document corresponding to the picked document image from a database, and then presenting additional information related to the electronic document to a user is disclosed in Takeda Kazutaka et al., “Real-time document image retrieval on a smartphone”, the Institute of Electronics, Information and Communication Engineers, Technical Report of IEICE, PRMU2011-104, pp. 31-36, 2011. A query of retrieval of the electronic document is a document image acquired from a camera mounted on the portable terminal. The additional information is displayed, for example, in a superimposed manner on a display mounted on the portable terminal. A method for displaying additional information on a projected paper document in a superimposed manner using a projector as a device which displays additional information is disclosed in Suzuki Wakana et al., “Addition of interactivity to printed document by information projection using a projector”, the Institute of Electronics, Information and Communication Engineers, technical Report of IEICE, PRMU2011-111, pp. 69-74, 2011.
In order not to impair usability, such as feeling of use and operability, by the user when performing the superimposed display of the additional information, a function to continuously display the additional information in the superimposed manner at an appropriate position is desired even in a case in which the relative positional relationship between a document of a paper medium and a camera is changed. In order to implement this function, a technique of tracking a document image for sequentially estimating the relative position and posture of the camera and the document of the paper medium is desired.
As a technique of tracking the document image, there is a method for implementing tracking by extracting characteristic pixel points (feature points), such as endpoints and crosspoints of characters, with respect to the document image, and tracking the positions of the feature points between adjacent frames. For example, feature points are extracted by extracting corner portions of an image using a method called the Harris operator, and the feature points are tracked through optical flow calculation in accordance with the Lucus-Kanade method. Then, the relative position and posture between the camera and the document of the paper medium are estimated on the basis of the entire tracking result of the feature points. A method for hourly updating the display position of the additional information using this estimation result has been disclosed.